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Abstract 


The  paper  consists  of  two  parts.  The  first  part  shows  that,  given  a  fam- 
ily of  k^d  compact  convex  and  separated  sets  in  E''  (i.e.  no  (ife-2)-flat  inter- 
sects all  these  sets),  the  space  of  all  oriented  common  tangent  hyperpianes  to 
these  sets,  which  separate  them  in  any  fixed  manner  (i.e.  contain  some  speci- 
fied subset  of  them  on  their  positive  side  and  the  rest  on  their  negative  side) 
is  homeomorphic  to  the  sphere  S''~^.  The  second  part  exploits  this  result  to 
obtain  sharp  botmds  on  the  combinatorial  complexity  of  the  space  of  all  com- 
mon transversal  hyperpianes  to  a  given  <f-separated  family  of  n  compact  con- 
vex sets  in  E'',  showing  this  complexity  to  be  0(n''~^). 
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1.  Introduction. 

The  purpose  of  this  paper  is  twofold:  to  prove  that  under  certain  circumstances  the 
space  of  common  tangent  hyperplanes  to  k  <  d  compact  convex  sets  in  R  has  a  simple 
topological  structure,  and  to  show  that  this  fact  can  be  used  to  bound  the  combinatorial 
complexity  of  the  set  of  hyperplanes  which  can  be  common  transversals  of  a  large  family 
of  separated  convex  sets.  This  then  allows  us,  in  turn,  to  derive  an  efficient  algorithm  for 
determining  the  space  of  common  transversals  if  the  convex  sets  are  polytopal  or,  more 
generally,  have  sufficiently  "simple"  boundaries. 

If  Jvi,  K2,  and  K3  are  three  disjoint  spheres  in  "general  position"  in  R"^  (i.e.,  not  all  met 
by  any  line),  and  T  is  the  family  of  common  tangent  planes  separating  them  in  a  given 
way,  say  the  first  two  from  the  third,  or  all  three  on  one  side,  then  it  is  not  difficult  to  see 
that  T  consists  of  precisely  two  members.  Similarly,  if  Ki  and  K2  are  two  disjoint  spheres 
in  R^  and  T  is  the  family  of  planes  tangent  to  A'l  and  A'2,  say  "internally",  it  is  clear  that 
T  consists  of  a  circle  of  tangent  planes.  If  we  clarify  the  situation  by  considering  instead 
oriented  planes  H  and  assigning  to  each  of  Ki^K2  a  sign  indicating  on  which  side  of  H  it 
is  to  lie,  this  circle,  or  the  pair  of  points  in  the  first  case,  has  a  natiiral  embedding  in  the 
sphere  S^  of  all  unit  normal  vectors  to  oriented  planes  in  R^ . 

In  this  paper  we  consider  to  begin  with  the  more  general  problem  in  which  the  dimension 
of  the  ambient  space  is  any  number  d  >  2,  and  the  spheres  are  replaced  hy  k  <  d  suitably 
separated  compact  convex  sets.  Our  resiilt  (Theorems  1  and  2  below)  is  that  if  Ai, . . . ,  Kk 
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axe  separated,  then  the  space  of  common  supporting  oriented  hyperplanes  div-iding  them 
in  a  given  way  is  homeomorphic  to  a  sphere  of  dimension  d  —  k. 

Recall  [5]  that  a  family  K  =  {A'l,. . . ,  A'^}  of  convex  sets  is  said  to  be  separated  in  R"^ 
if  it  has  no  {d  —  2)-traiisversed  (i.e.,  if  there  is  no  {d  —  2)-flat  meeting  all  the  members  of 
K),  or  equivaJently  if  any  r  of  the  sets  can  be  sepairated  from  the  remaining  d  —  r  by  a 
hyperplane;  here  we  shedl  extend  this  to  a  family  K  of  ^"  <  d  +  1  sets  by  requiring  that 
the  members  of  K  have  no  (k  —  2)-trajisversal.  (Thus  two  sets  ase  separated  if  they  have 
no  point  in  common,  three  if  they  are  met  by  no  line,  and  so  on.)  It  turns  out  that  this 
is  precisely  the  notion  of  "general  position"  needed  to  generalize  disjointness  (in  the  case 
of  two  convex  sets)  which  permits  us  to  extend  the  results  mentioned  above  for  spheres  to 
convex  sets  in  higher  dimensions. 

Surprisingly,  the  only  work  relating  to  this  result  which  appears  in  the  literature  seems 
to  be  a  paper  by  H.  Brunn  [2],  which  is  restricted  to  the  case  of  two  convex  bodies  in  R"*, 
and  from  which  our  results  can  easily  be  seen  to  follow  in  that  specied  low-dimensionaJ 


case. 


In  Section  3  we  apply  the  results  of  Section  2  on  the  topology  of  the  space  of  common 
tangent  hyperplanes  to  derive  an  asymptotic  bound  on  the  complexity  of  the  space  r(K)  of 
common  transversal  hyperplanes  to  a  collection  K  of  n  (f-wise  separated  compact  convex 
sets  in  R'',  for  n  large.  This  problem  has  been  considered  previously  in  the  computer 
science  literature,  but  only  in  the  special  cases  of  dimension  2  (see  [4]  for  example,  where 
a  tight  bound  of  2n  —  2  was  obtained  on  the  complexity  of  r(K))  and  dimension  3  (see  [3] 
and  Remark  (3)  in  Section  4  below). 

Here  we  axe  able  to  show  (Theorem  3)  that  for  J  >  2  the  complexity  of  r(K)  is  0(n''~' ), 
which  extends  the  results  previously  known  in  the  lower-dimensional  cases.  In  particular-, 
we  show  that  this  bound  applies  as  well  to  the  number  of  distinct  order  types  in  which  a 
transversal  may  meet  the  members  of  a  given  family  of  separated  sets,  a  problem  which 
(in  dimension  2  at  least)  has  drawn  some  recent  attention  ([4],  [7],  [11]). 
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Finally,  in  Section  4,  we  present  an  algorithm  which  uses  the  analysis  of  the  preceding 
sections  to  calciolate  efficiently  the  complete  space  of  hyperplanes  transversal  to  a  given 
family  of  separated  compact  convex  sets  in  R'',  provided  the  sets  themselves  are  described 
in  a  way  which  permits  "constant-time"  manipulation  of  any  fixed  number  of  them,  e.g., 
if  each  set  is  a  convex  polytope  with  a  bounded  number  of  sides.  If  there  are  n  sets,  the 
space  of  common  transversals  can  be  calculated  in  time  0{n  ~^  log   n). 

Section  4  concludes  with  a  few  remarks  on  open  problems. 

We  would  like  to  express  our  gratitude  to  Janos  Pach  for  some  helpful  discussions,  and 
to  acknowledge  that  Tibor  Bisztriczky  has  independently  found  a  proof  of  Theorem  2  in 
the  special  case  A;  =  d  [1],  along  somewhat  different  lines  from  ours. 

2.  The  space  of  common  tangents. 

Lemma  1.  Let  K  =  {Ko,Ki^ . . . ,  Kd}  be  a  separated  family  of  compact  convex  sets  in  R^ 
with  Kq  a  point,  and  let  K  =  P  U  N  be  a  paxtition  of  K  into  two  disjoint  subsets.  Suppose 
there  is  an  oriented  hyperplane  H  which  has  the  members  of  P  on  its  positive  side  and 
those  of  N  on  its  negative  side,  and  which  supports  A'l , . . . ,  K^.  Then  H  is  unique. 

Proof:  Suppose  H  and  H'  axe  two  such  oriented  hyperplanes.  Let  F  be  the  (d  —  2)-flat 
in  which  they  meet,  and  assume  to  begin  with  that  F  is  at  finite  distance.  Together,  H 
and  H'  divide  R  into  four  quadrants,  of  which  one,  call  it  Qi,  lies  on  the  positive  side  of 
both  H  and  H' ,  and  the  opposite  one,  say  Q2,  on  the  negative  side  of  both.  It  is  clear 
that  the  Ki  in  P  lie  in  Qi,  while  those  in  N  lie  in  Q2-  Rotating  H  about  F  through  Qi 
and  Q2  simultaneously  until  it  coincides  with  H',  H  passes  through  all  of  A'l,. . .  ,  A'j  at 
every  stage,  hence  never  through  A'o  (by  the  separation  condition).  But  at  the  end,  all  of 
A'l , ... ,  Ki  lie  on  the  wrong  side  of  the  rotated  H ,  hence — reversing  the  orientation  to  get 
H' — we  see  that  H'  cannot  have  A'o  on  the  correct  side. 

If  H  and  H'  are  parallel,  we  need  only  move  H  parallel  to  itself  until  it  coincides  with 
H' ,  and  then  the  rest  of  the  proof  goes  through.  D 


Lemma  2  (projection  lemma).  I£Ki, . . .  ,Kk  form  a  separated  family  ofk  <  d  compact 
convex  sets  in  R^,  then  so  do  their  orthogoned  projections  into  some  {k  —  I) -dimensional 
subspace  of  R"^. 

Proof:  Enlarging  Ccich  Ki  slightly  if  necessary,  we  may  assume  that  the  sets  A',  are 
strictly  convex  and  have  non-empty  interior. 

We  begin  by  proving  the  result  first  in  the  case  where  k  =  d,  and  then  indicate  the  modi- 
fications necessary  for  k  <  d.  For  this  purpose,  we  use  some  of  the  machinery  developed  in 
[5],  which,  in  turn,  was  inspired  by  [6].  For  each  point  x  at  infinity,  i.e.,  each  direction  of 
parallel  projection,  let  tTi  be  the  projection  mapping  of  R**  onto  the  hyperplane  H^  normal 
to  the  direction  represented  by  x  and  passing  through  the  origin.  We  seek  a  point  x  at 
infinity  such  that  7ri(A'i), . . . ,  ^^{Kd)  eire  separated,  i.e.,  such  that  x  is  not  co-hyperplanar 
with  any  choice  of  points  j/i , . . . ,  j/d,  y,  G  Ki. 

Suppose  there  is  no  such  point  x.  Fix  an  interior  point  r,  6  Ki  for  each  i,  and — for  every 
t  e  [0, 1] — let  Ki{t)  be  the  contraction  of  A",  by  the  factor  t  about  z,.  Since  A'l , . . . ,  A'j  are 
separated,  the  points  Zj , . . . ,  zj  are  in  genercd  position,  hence  we  can  certainly  find  a  point 
i(0)  at  infinity  not  co-hyperplanar  with  Ari(O), . . . ,  A'd(O).  As  t  increases  from  0  to  1,  it 
gets  progressively  harder  to  find  a  point  x{t)  not  co-hyperplanar  with  K\{t), . . .  ,Kd[t), 
i.e.,  the  values  of  t  for  which  this  is  possible  form  cm  interval  of  the  form  [0.  r),  r  <  1, 
which  in  fact  must  be  open  at  the  right  since  any  projection  which  keeps  the  images  of  the 
sets  Ki{t)  separated  does  the  same  for  the  sets  Ki{t-\-e).  Since  the  hyperplane  at  infinity  is 
compact,  we  can  find  an  increasing  sequence  <i,t2i  ■  •  •  converging  to  r  and  corresponding 
points  ii,  X2,  ■  ■  ■  converging  to  a  point  x  such  that  Xn  is  not  co-hyperplanar  with  the  sets 
Ki{tn)  for  each  n  =  1,2, . . . .  Further,  by  rotating  R*^  appropriately  for  each  n,  we  may 
assume  that  !„  =  i  for  each  n,  i.e.,  that  we  have  a  sequence  A'", . . . ,  A'^  of  families  of 
sets  converging  (in  the  Hausdorff  metric)  to  A'i(r), . . . ,  A'd(r),  such  that  for  each  n  the 
corresponding  sets  are  not  co-hyperplanar  with  i,  while  in  the  limit  they  are. 

Since  TTx  does  not  leave  the  sets  A',(r )  separated,  there  is  a  hyperplcme  H  through  x  which 


meets  all  the  sets  Ki{T).  It  is  clear  that  H  must  support  them — otherwise  we  could  rotate 
H  slightly  to  a  hyperplane  H'  still  passing  through  x  and  cutting  through  the  interiors  of 
all  of  the  sets  Ki(T),  and  then  we  could  find  a  family  A'", . . . ,  A'J  sufficiently  close  which 
H'  would  still  meet,  which  is  impossible.  Let  yi  =  H  d  A',(r)  for  each  i  =  1,. ..  ,d.  We 
claim  that  their  projections  7rj;(j/,)  on  Hx  form  a  minimal  Radon  partition  on  ^  fl  Hi  [5], 
i.e.,  that  those  corresponding  to  A'.'s  lying  on  one  side  of  H  carmot  be  separated  from 
those  corresponding  to  Ar,'s  lying  on  the  other  side.  This  follows  as  in  [5,  p.  306];  in  brief, 
the  reason  is  that  otherwise  we  could  rotate  H  slightly  to  cut  through  the  interior  of  all  of 
the  sets  A'i(r),  which  we  have  seen  above  is  impossible.  Since  there  are  fewer  than  d  +  I 
points  of  contact  y,,  we  can  rotate  H  slightly  to  move  it  away  simultaneously  from  all  of 
the  sets  Ar,(r),  so  that  some  appropriate  projection  tTj./  (with  x'  close  to  x)  moves  the  sets 
Ar,(r)  away  from  H  D  Hx  on  both  sides.  But  then  it  follows  from  Lemma  3  of  [5]  that  no 
other  (c/— 2)-flat  can  meet  the  sets  7rj;'(A'i(T)), . . .  ,7ri/(A'(i(r)),  which  gives  a  contradiction. 

U  k  <  d,  we  make  the  following  modifications:  x  should  now  be  thought  of  as  a  (c?  —  k)- 
flat  at  infinity,  and  tTj  as  the  corresponding  projection  onto  the  (k  —  l)-space  orthogonal 
to  the  {d  —  k  +  l)-space  spanned  by  x.  It  is  now  the  compactness  of  the  Grassmannian 
Gd-k+i,d  of  subspaces  of  R**  of  dimension  d—  k  +  1  which  we  invoke  to  find  our  convergent 
sequences,  and  the  rest  of  the  details  remain  as  above.  D 

Theorem  1.  Let  K  =  {ATo,  ATi, . . . ,  Kd}  be  a  separated  family  of  compact  convex  sets  in 
R^  with  Kq  a  point,  and  iet  K  =  P  U  N  be  a  partition  of  K  into  two  disjoint  subsets. 
Then  there  exists  one  and  only  one  oriented  hyperplane  H  which  has  the  members  of  P 
on  its  positive  side  and  those  of  N  on  its  negative  side,  and  which  supports  A'l, . . . ,  Kd- 
Moreover,  H  varies  continuously  with  A'o, . . .  ,Kd  in  the  Hausdorff  topology,  as  long  as 
these  sets  remain  separated. 

Proof:  The  uniqueness  has  already  been  proved  in  Lemma  1.  For  the  existence  we 
proceed  by  induction  on  d,  the  assertion  being  clear  for  d  =  1.    Assuming  d  >  2,  let 
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TTi :  R''  — »  R'^"^  be  a  separating  projection  for  /Ci, . . . ,  Kj,  as  in  Lemma  2;  i.e.,  the  images 
'^x{Ki)y  •  •  •  1  "^ziKd)  in  R''"^  have  no  transversal  (ti— 2)-flat.  By  induction  hypothesis,  there 
is  a  unique  oriented  ((i  —  2)-flat  F  in  R*^"^  separating  those  sets  (with  their  inherited  signs) 
correctly,  and  supporting  ■^^{Ki),. . .  ,-Kx{Kd-\)-  (Notice  that  since  7rj;(i\i ),...,  7ri(ivd) 
axe  separated,  it  maJces  no  difference  which  point  of  T^z{Kd)  we  choose  to  play  the  role 
of  the  "ivo"  in  the  induction  hypothesis.)  Also,  again  by  induction  hypothesis,  there  is  a 
second  oriented  {d  —  2)-flat  F'  with  the  same  properties  as  F,  except  that  it  has  7ri(A'd) 
on  its  "wrong"  side.  Lifting  these  up  to  R**  by  7rj\  we  get  oriented  hyperplanes  H  and  H' 
each  supporting  ATi , . . . ,  Kd-i,  with  H  separating  A'l , . . . ,  /Cj  correctly,  and  H'  separating 
ATi, . . . ,  Kd-\  correctly  but  having  K^  on  its  wrong  side. 

Again  by  Lemma  2,  there  is  a  separating  projection  tTj,:  R  ~'    — >  R  ~^  for  the  sets 
7rj(A'i), . . . ,  7rj;(A'd_i).  This  implies  that  the  map 

TT^  =  TTj,  o  TT^  :  R"*  -*  R''-^ 

is  a  separating  projection  for  the  fcimily  {K\, . . . ,  Kd-\ },  where  L  =  7rJ^(j/),  so  that  as  z 
varies  continuously  in  L  we  get  a  continuously  varying  circle  of  projections  tt.  onto  an  R 
(which  we  may  regard  as  remaining  fixed),  such  that  each  fajnily  {;r-(A']), ....  7r.(A'd-i)} 
is  separated.  These  projected  sets  {7rj(A',)}  vary  continuously  with  r,  so  by  induction 
hypothesis  the  corresponding  supporting  hyperplanes  F-  in  R  edso  vary  continuously; 
hence  as  their  pullback  H,  =  n~^{F^)  to  R''  moves  from  H  to  H'  (which  corresponds  to 
z  moving  along  L  from  x  to  "— i")  there  must  be  at  least  one  position  H\  in  which  H. 
separates  all  of  A"], ...  ,  A'j  correctly,  and  supports  A'j  as  well,  and  a  second  as  z  continues 
along  L  back  to  x  again,  say  H2,  in  which  H,  does  the  same.  By  Lemma  1,  however,  at 
most  one  of  these  has  A'o  on  its  positive  side,  and  at  most  one  on  its  negative  side.  Hence 
at  least  one  must  have  A'o  on  its  correct  side. 

Finally,  we  can  see  that  the  common  tangent  hyperplane  H  varies  continuously  with 
Kq,.  . .  ,Kd  as  follows:    For  each  oriented  hyperplane  H  with  positive  normal  vector  u 
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having  Kq  on  its  correct  side,  let  (i//,i  be  the  distance  H  must  be  moved  in  a  normai 
direction  to  support  Ki  on  the  correct  side,  and  let 

«=1 
Then  we  have  been  seeking,  and  have  foimd,  a  unique  hyperplane  H  which  minimizes  the 

continuous  function  Dh  over  an  appropriate  compact  subset  of  the  space  of  all  hyperplanes 

in  R"*.   (In  fact,  this  minimum  has  tiirned  out  to  be  0!)   But  since  the  function  Dh  itself 

depends  continuously  on  Kq,  . . . ,  Kd  as  well  as  on  H,  so  does  the  hyperplane  achieving  the 

minimum,  by  the  uniqueness  and  the  compactness.  This  concludes  the  proof.  D 

Remark:  It  is  also  possible  to  prove  Theorem  1  along  the  Unes  suggested  in  the  last 
paragraph:  find  a  hyperplane  H  which  minimizes  the  function  Dh,  and  prove  it  must 
satisfy  the  requirements  of  the  theorem  by  rotating  it  slightly  to  reduce  Dh  if  it  does  not; 
the  details  axe  somewhat  more  intricate  than  those  above. 

Notice  also  that  if  several  of  the  Ki  happen  to  be  points,  even  points  at  infinity,  the 
conclusion  of  Theorem  1  remains  valid,  since  we  can  always  move  them  to  finite  distance 
by  a  permissible  projective  transformation,  i.e.,  we  can  move  the  hyperplane  at  infinity 
away  from  them  without  affecting  the  remaining  convex  bodies  in  any  material  way.  (Of 
course  if  some  Ki,  1  <  i  <  d,  is  a.  point,  the  "positive"  and  "negative"  side  condition  for 
it  is  vacuous.) 

Theorem  2.  Let  K  =  {Ki , . . . ,  Kk},  k  <  d,  be  a  separated  family  of  compact  convex  sets 
in  W^,  and  iet  K  =  P  U  N  be  a  partition  of  K  into  two  disjoint  subsets.  Let  T  be  the  set 
of  oriented  hyperplanes  H  which  have  the  members  of  P  on  their  positive  side  and  those 
of  N  on  their  negative  side,  and  which  support  Ki, . . . ,  Kk.  Then  T  (with  its  members 
identified  with  their  positive  normals  in  S^~^)  is  a  topological  sphere  of  dimension  d  —  k. 

Proof:  Theorem  1  disposes  of  the  case  k  =  d;  let  us  therefore  suppose  that  k  <  d. 
By  Lemma  2  we  can  find  a  {d  —  /:)-flat  F  at  infinity  such  that  for  any  choice  of  Xi  G 
Ki,  . . . ,  Xk  ^  Kk,  the  Xi  and  F  are  in  general  position. 
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Consider  a  (d— fc  — l)-flat  G  lying  in  F,  choose  an  orientation  for  G,  i.e.  a  ''positive"  side  of 
G  in  F,  and  choose  d—k  points  Pjt-t-i , . .  . ,  Pj  in  genereJ  position  in  G.  By  Theorem  1  and  the 
Remark  following,  there  is  one  and  only  one  oriented  hyperpleine  H  supporting  /Ci, . . . , Kk 
correctly,  passing  through  Pit+i ,...,  Pd  (hence  cutting  F  in  G),  and  determining  the  same 
orientation  in  F  as  does  G.  (Intuitively,  H  is  the  unique  member  of  T  whose  normal  is 
orthogonal  to  G  and  points  to  the  positive  side  of  G.)  If  we  reverse  the  orientation  of 
G  then  the  oriented  hyperplane  H  changes,  while  if  we  move  G  to  a  new  position  H  also 
moves,  since  otherwise  H  would  meet  F  in  two  distinct  {d—k  —  l)-flats  and  would  therefore 
contain  F  entirely,  contrary  to  the  general  position  assumption. 

On  the  other  hemd,  any  hyperplane  H  which  supports  Ki, . . . ,  Kk  and  sepeirates  them 
correctly  must  cut  F  in  some  (d  —  k  —  l)-flat  G,  so  there  is  a  bijection  /  from  the  set 
of  oriented  (d  —  k  —  l)-flats  contained  in  F  into  the  set  T  of  hyperplanes  supporting 
Ki, . . . , Kk  and  separating  them  correctly.  Moreover,  /  is  continuous  by  Theorem  1.  Since 
the  (d  —  k  —  l)-fiats  contained  in  F  form  a  projective  space  P'^~''  of  dimension  d  —  k,  sind 
the  specification  of  an  orientation  for  each  fiat  is  just  the  lift  of  P'^~'=  to  its  covering  space 
S'^~''  (recall  that  P'^~''  is  obtained  by  identifying  emtipodal  points  of  5"*^"*),  the  conclusion 

follows.  n 

Remark:  If  K  consists  simply  of  k  points  in  general  position,  the  sphere  T  given  by 
Theorem  2  reduces  to  the  intersection  of  S'^~^  with  the  [d  —  k  +  l)-space  normal  to  the 
aifine  span  of  K.  By  deforming  the  sets  Ki  in  Theorem  2  continuously  to  points,  and 
applying  the  theorem  at  each  stage,  one  can  then  show  that  the  pair  {S'^~^,T)  is  isotopic 
to  the  standard  pair  (S''"^  5''"*),  so  that — in  particular — T  is  always  unknotted  in  the 
ambient  sphere  S**"'. 

3.  The  space  of  common  transversals. 

Let  K  =  {A'l, . . . ,  K„}  be  a  collection  of  n  >>  d  compact  convex  sets  in  R*^,  any  d  of 
which  are  separated.  A  hyperplane  H  is  a  common  transversal  {or  a  stabbing  hyperplane)  of 
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K  if  i/Tliir,  7^  0  for  all  z  =  1, . . . ,  n.  In  this  section  we  apply  the  previous  results  to  analyze 
the  combinatorial  structure  of  the  space  r(K)  of  all  common  transversal  hyperplanes  of 
K,  and  in  particular  to  bound  its  complexity. 

To  define  r(K)  more  precisely,  we  represent  each  oriented  hyperplane  if  by  a  pair  (u,  d), 
where  u  €  S^~^  is  a  imit  vector  in  the  direction  of  the  normal  to  H,  and  d  is  the  signed 
distance  from  the  origin  to  H,  i.e.,  d  =  (a,  u)  for  any  point  a  on  H;  notice  that  (u,  d)  and 
(  — u,  — (i)  are  the  two  oriented  representations  of  the  same  unoriented  hyperplane.  The 
space  r(K)  is  thus  the  subset  of  S'^~^  x  R  consisting  of  all  pairs  (u,(f)  representing  the 
common  transversal  hyperplanes  to  K. 

To  get  a  better  handle  on  the  structure  of  r(K),  we  define,  for  each  i  =  1,. . .  ,n,  a 

function  /,  on  the  unit  sphere  5*^"^  as  follows.    For  each  vector  u  6  5"^"^  let  /i(u)  be 

the  signed  distance  from  the  origin  to  the  hyperplane  supporting  Kj  and  having  positive 

normal  u.  Let 

/(u)  =  min/,(u),      ueS''-'. 
t 

Let  H  he  a,  common  transversal  hyperplane  of  K  and  choose  an  orientation  so  that  H  has 

some  normal  direction  u.  Then  the  signed  distance  d  oi  H  from  the  origin  (measured,  as 

above,  by  (a,  u),  where  a  is  any  point  on  H)  must  satisfy  d  <  fi{u)  for  all  i,  or  d  <  /(u). 

Similarly,  if  we  taJce  — u  as  the  normal  direction  to  H ,  we  obtain  —d  <  /(  — u).   Thus  we 

must  have 

/(u)  +  /(-u)>0. 

It  is  also  easily  checked  that  any  u  G  S'^~^  that  satisfies  this  inequality  admits  a  common 
transversal  hyperplane  to  K  orthogonal  to  u.  Furthermore,  the  desired  space  t(K)  can  be 
represented  as 

r(K)  =  {(u,  d):ue  S"-',  (i  €  R,  /(u)  +  /(-u)  >  0,  -/(-u)  <d<  /(u)}. 

In  particular,  complete  information  about  r(K)  is  provided  by  its  projection  <7(K)  on 
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5''-\i.e., 

a{K)  =  {ue  S''-'  :  /(u)  +  /(-u)  >  0}, 

together  with  a  pair  of  partitions  of  this  space,  denoted  by  <7'''(K)  cind  <7~(K),  into  maximal 
connected  cells  of  dimensions  between  0  and  d  —  I,  so  that  to  eax:h  cell  in  a''"(K)  (resp. 
cr"(K))  there  corresponds  a  fixed  known  function  fi(u)  (resp.  fj{  —  u))  (or  a  fixed  set  of 
such  functions)  which  attains  the  minimum  /(u)  (resp.  /(  — u)).  Our  goal  is  to  analyze 
the  combmatoTial  complexity  of  (7"'"(K)  and  of  a~(K),  namely  the  number  of  cells  in  these 
two  partitions. 

To  analyze  the  combinatorial  structure  of  cr"''(K),  say,  we  proceed  as  foUows.    Fix  an 
index  i  =  1, . . . ,  n.  For  each  j  ^  i  consider  the  following  two  cells  on  S'^~^ : 

Rf  =  {ue  S'-'  :  fj(u)  <  /,(u)} 

R-  =  {ueS''-':fj(-u)  +  Mu)<0}. 

(Intuitively,  Rj  consists  of  those  directions  in  which  a  sweep-hyperpleine  leaves  Kj  before 
leaving  K,,  while  Rj  consists  of  those  directions  in  which  it  leaves  A',  before  entering  Kj.) 
Let 

R  =  R('^  =  \J{R+UR-). 

Then  clearly  u  G  S'^-'\i?  if  and  only  if /(u)  +  /(-u)  >  Oand/(u)  =  /,(u).  In  other  words, 
the  complement  of  R  is  exactly  that  portion  of  cr''"(K)  where  the  minimvun,  /,  is  attained 
by  /,.  It  thus  suffices  to  determine  an  upper  bound  on  the  combinatorial  complexity  of 
the  boundary  of  R;  multiplying  this  bound  by  n  will  then  yield  a  bound  on  the  complexity 

of  <7+(K). 

Lemma  3.  The  set  {dR'j' }  j:^,U{dRj  }j^i  is  an  arrangement  of  pseudo-spheres  withui  S^~^ 
(i.e.,  the  intersection  of  any  subcollection  of  \  <  k  <  d  of  these  objects  is  homeomorphic 
to  the  unit  sphere  S''~*~',  just  as  would  be  the  case  for  an  arrangement  of  great  spheres 
in  general  position). 
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Proof:  Let 

{dRt;,...,dR+,dR-,...,dR-} 

be  a  subcollection  of  A;  =  p  +  q  of  these  boundaries.  It  follows  from  the  definition  that 
their  intersection  is  the  subset  of  5**"^  consisting  of  the  normal  directions  u  of  all  oriented 
hyperplanes  H  such  that  H  is  tangent  to  Ki,  Ki^ , . . . ,  Ki^ ,  Kj^ ,  ■  •  • ,  Kj^ ,  and  such  that  the 
first  p  + 1  sets  lie  on  the  negative  side  of  H  and  the  last  q  on  the  positive  side.  By  Theorem 
2,  this  space  is  homeomorphic  to  S^~''~^,  as  asserted.  D 

The  following  proposition  follows  immediately  from  the  reults  of  [8]  (see  also  [9]  and 
[10]),  but  we  include  a  sketch  of  the  proof  for  the  sake  of  completeness. 

Proposition  1.  The  combinatorial  complexity  of  the  boundary  of  the  union  ofn  pseudo- 
hemispheres  (i.e.,  sets  whose  boundaries  constitute  an  arrangement  of  pseudo-spheres  in 
the  sense  de£ned  above)  in  S^~^ ,  for  d  >  Z,  is  0{n^~^). 

Proof:  We  use  induction  on  d.  The  base  case  c?  =  3  is  merely  a  special  case  of  the  result 
of  [8]  that  the  union  of  n  Jordan  regions  in  the  plane,  bounded  by  Jordan  curves  with  the 
property  that  each  pair  of  them  intersects  at  most  twice  (exactly  twice  in  our  case),  has 
only  a  linear  number  of  such  intersections  on  its  boundary.  For  cf  >  3  we  apply  the  same 
idea  as  above:  Let  Bi,. . .  ,Bn  be  the  given  pseudo- hemispheres.  We  fix  one  of  them,  say 
Bi,  and  consider  the  intersections  Ci  =  dBi  f]Bi,ior  i  ^  1.  It  is  easily  checked  that  the  C^'s 
form  a  collection  of  pseudo-hemispheres  within  5Si,  which  is  homeomorphic  to  S^~^.  By 
induction  hypothesis,  the  boundary  of  the  union  of  the  C^'s  has  combinatorial  complexity 
0(n'' "■■').  But  this  boundary  is  plainly  equal  to  the  boundary  of  that  portion  of  dBi 
which  appears  along  the  boundary  of  the  union  of  the  B,'s.  Thus  the  total  combinatorial 
complexity  of  the  boundary  of  this  union  is  at  most  n  •  0{n'^~^)  =  0{n^~^).,  as  claimed.  D 

As  a  corollary,  we  immediately  obtain  our  main  result: 

Theorem  3.  The  combinatorial  complexity  of  (7"^(K)  ancf  of  cr~(K),  hence  also  of  r(K), 
isO{n'^-^). 
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4.  Algorithms  and  remarks. 

Assuming  that  the  sets  Ki  have  relatively  simple  shape  so  that  manipulating  any  fixed 
number  of  the  functions  /,(u)  can  be  assumed  to  taJce  constant  time,  we  can  calculate  r(K) 
efficiently.  The  method  follows  the  line  of  proof  of  Proposition  1,  tracing  the  induction 
down  to  dimension  2,  at  which  point  it  applies  the  edgorithm  given  in  [8].  The  entire 
procedure  takes  time  0{n'^~^  log    n). 

To  avoid  the  rather  tedious  details  of  calculating  the  topological  structvue  of  7-(K),  we 
restrict  the  algorithm  to  produce  only  the  vertices  of  that  set,  i.e.,  only  extreme  common 
transversal  hyperplanes  that  axe  also  common  tangents  to  d  of  the  given  sets. 

Algorithm:  We  first  describe  the  inner  routine  of  the  algorithm,  which  recurses  on  the 
dimension,  and  then  give  the  top-level  part. 

Procedure  UNION: 

Input:  A  dimension  r  >  2  and  a  set  of  pseudo-hemispheres  Bi, . . .  ,Bq  on  the  sphere  S'^. 

Output:  The  vertices  of  |J.  Bj. 

if  r  =  2  then 

Bi, . . . ,  Bg  aTe  "pseudo-disks"  on  S^ . 

Apply  the  divide-and-conquer  technique  of  [8]  to  calculate  the  desired  vertices 

This  assumes  that  the  Bj  's  have  some  reasonable  m,onotonicity  properties,   to  allow 
us  to  apply  line  sweeping.    (This  is  the  case,  for  exam,ple,  if  each  B,   is  actually  a 
hem,isphere,  which  happens  if  all  the  original  sets  Ki  are  balls.)   The  algorithm  takes 
0{q  log    q)  time. 
else 

for  each  i  =  I, . . .  ,q 

for  each  j  7^  i  let  B'  —dB.D  Bj 

call  UNION  with  r  -  I  and  {B^  :  j  ^  i} 

The  Bj   are  pseudo-hemispheres  on  the  (r  —  l)-sphere  dBi. 
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endfor 

Return  the  union  of  the  outputs  of  all  recursive  calls,  with  duplications  removed 
Each  vertex  is  ■produced  r  times. 
endif 
end  UNION 

MAIN  ROUTINE: 

Input:  The  dimension  d  and  a  family  K  of  n  rf-wise  separated  convex  sets  in  R  . 
Output:  The  collection  of  all  extreme  common  transversed  hyperplanes  of  K. 
for  each  i  =  1, . . . ,  n 

Form  the  pseudo- hemispheres  i?+    Rj  for  all  j  ^  i 

call  UNION  with  d  —  1  and  the  set  of  these  pseudo-hemispheres 
endfor 
Return  the  union  of  the  outputs  of  all  calls  to  UNION,  with  duplications  removed 

Each  vertex  is  produced  d  times. 
end 

Every  vertex  is  an  extreme  common  transversal,  tangent  to  those  d  sets  Ki  for  which 
the  top-level  call  to  UNION  has  produced  that  vertex. 

The  time  complexity  of  the  algorithm  is  an  immediate  consequence  of  Proposition  1,  the 
discussion  preceding  it,  and  the  time  complexity  of  the  innermost  stage  of  the  algorithm, 
as  borrowed  from  [8].  Notice  that  the  only  place  in  the  algorithm  where  the  actual  shape 
of  the  sets  Ki  really  matters  is  at  the  bottom  of  the  recursion,  at  r  =  2.  In  other  words, 
we  need  to  assume  only  that  the  shape  of  each  pseudo-circle  of  oriented  hyperplanes  that 
are  tangent  to  any  fixed  d—\  sets  in  K  and  separate  them  in  any  specific  manner  is  simple 
enough  for  the  algorithm  of  [8]  to  be  applicable. 

Remarks: 

(1)  An  open  problem  is  whether  the  bound  provided  by  Theorem  3  is  tight  in  the  worst 
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case,  for  separated  convex  sets.  It  is  easy  to  construct  an  example  where  the  complexity  of 
r(K)  is  fi(n'-7-' ),  as  follows.  Assume  for  simplicity  that  d  is  a  power  of  2.  For  the  base  case 
d  =  2  use  the  construction  in  [4].  For  d  >  2,  taJce  two  complement axy  subspaces  Vi,  V2  of 
R**  of  dimension  d/2  each.  Construct  inductively  in  each  of  them  a  family  Ki,  K2  of  n/2 
separated  convex  sets  whose  "stabbing  region"  has  combinatorial  complexity  fi(u<),  ajid 
observe  that  any  pair  of  hyperplanes  (in  Vi ,  V2  respectively)  which  Eire  common  transversals 
in  Kj,  K2,  respectively,  spem  a  hyperplajie  in  R**  which  is  a  common  transversal  to  the 
separated  family  K  =  Ki  U  K2.  This  implies  easily  that  the  combinatorial  complexity  of 
r(K)  is  at  least  17((n<)'^)  =  Q(n2 ),  the  constant  depending  on  d.  It  is  easy  to  extend  this 
construction  to  the  case  of  general  d.  Can  this  lower  bound  be  improved?  Alternatively, 
can  the  upper  bound  of  Theorem  3  be  improved? 

(2)  It  is  instructive  to  notice  that  the  analysis  above  also  provides  a  means  to  study  the 
combinatorial  complexity  of  the  convex  hull  of  K.  Indeed,  for  this  purpose  we  want  to 
consider  the  space  of  all  hyperplanes  that  contain  all  the  sets  in  K  on  the  same  side.  But 
the  boundary  of  this  space  is  easily  seen  to  coincide  with  the  upper  envelope  of  the  graphs 
of  the  functions  /,(u),  as  defined  above,  rather  than  with  their  lower  envelope  /.  Thus  our 
analysis  allows  us  to  conclude  that  the  combinatorial  complexity  of  the  convex  hull  of  K  is 
0{n'^~^).  Generally  speaking,  there  does  not  exist  complete  symmetry  between  these  two 
problems,  because  the  shape  of  the  functions  /,  seems  to  render  the  structure  of  their  upper 
envelope  simpler  than  that  of  their  lower  envelope.  Nonetheless,  this  connection  between 
convex  hulls  and  common  transversals  is  intriguing,  aind  raises  several  open  problems,  the 
most  obvious  being  to  obtain  sharp  bounds  on  the  complexity  of  the  convex  hull  of  n 
(d-wise  separated)  convex  sets  in  R''.  If  these  sets  reduce  to  points,  the  complexity  of  the 
hull  is  at  most  0(nl-2-'),  as  implied  by  the  "upper  bound  theorem"  for  convex  polytopes. 
Is  this  also  an  upper  bound  for  the  case  where  the  sets  in  K  are  larger  than  points  (but 
still  separated)?  See  [10]  for  a  discussion  of  some  special  cases  of  the  convex  hull  problem 
for  sets  in  three  dimensions. 
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(3)  If  the  sets  in  K  are  not  separated  then  in  general  very  little  can  be  said  about  the 
complexity  of  r(K).  For  example,  if  two  convex  sets  are  allowed  to  overlap,  even  the  space 
of  all  common  tangent  hyperplanes  to  both  sets  can  be  of  arbitrarily  large  combinatorial 
complexity.  In  certain  non-separated  cases,  however,  sharp  bounds  can  still  be  obtained 
for  the  complexity  of  t(K).  For  example,  if  the  sets  in  K  are  all  convex  polytopes  in 
three  dimensions  (not  necessarily  separated),  then  it  is  shown  in  [3]  that  the  combinatorial 
complexity  of  t(K.)  is  0{n^a{n)),  where  n  is  the  total  number  of  vertices  of  these  polytopes 
and  where  a{n)  is  the  extremely  slowly  growing  inverse  Ackermann  function.  The  bound 
reduces  to  only  0{n^)  for  a  collection  of  n  line  segments  in  3-space. 

(4)  In  addition  to  the  combinatorial  complexity  of  t(K),  there  are  two  other  parameters 
of  interest.  One  is  the  number  i^(K)  of  connected  components  of  t(K),  and  the  other 
is  the  number  /x(K)  of  different  order  types  that  can  be  induced  by  common  transversal 
hyperplanes  of  K,  where  the  order  type  induced  by  a  transversal  hyperplane  H  in  R  is 
determined  by  choosing  a  point  in  each  intersection  H  0  Ki,  i  =  1, . . . ,  n,  and  reading  off 
the  orientations  of  all  c?-tuples  of  points  selected.  Since  the  sets  in  K  are  separated,  it  is 
easily  seen  that  the  order  type  induced  by  H  is  uniquely  defined,  and  does  not  change  as 
H  varies  in  a  single  connected  component  of  r(K),  so  that  n{K)  <  i^(K);  and  it  is  equally 
clear  that  i^(K)  does  not  exceed  the  combinatorial  complexity  of  t(K).  Thus  the  upper 
bound  established  in  Theorem  3  applies  to  //(K)  and  to  i^(K)  as  well.  In  R'^,  moreover, 
the  results  of  [4]  show  that  the  maximum  possible  values  of  all  three  parameters  are  the 
same  (namely  2n  —  2  for  n  sets).  Does  a  similar  equality  hold  in  higher  dimensions? 
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